Goto

Collaborating Authors

 Java Sea


LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models

Wang, Renzhi, Li, Piji

arXiv.org Artificial Intelligence

Large language models (LLMs) require continual knowledge updates to stay abreast of the ever-changing world facts, prompting the formulation of lifelong model editing task. While recent years have witnessed the development of various techniques for single and batch editing, these methods either fail to apply or perform sub-optimally when faced with lifelong editing. In this paper, we introduce LEMoE, an advanced Mixture of Experts (MoE) adaptor for lifelong model editing. We first analyze the factors influencing the effectiveness of conventional MoE adaptor in lifelong editing, including catastrophic forgetting, inconsistent routing and order sensitivity. Based on these insights, we propose a tailored module insertion method to achieve lifelong editing, incorporating a novel KV anchor routing to enhance routing consistency between training and inference stage, along with a concise yet effective clustering-based editing order planning. Experimental results demonstrate the effectiveness of our method in lifelong editing, surpassing previous model editing techniques while maintaining outstanding performance in batch editing task. Our code will be available.


The Case for Scalable, Data-Driven Theory: A Paradigm for Scientific Progress in NLP

Michael, Julian

arXiv.org Artificial Intelligence

I propose a paradigm for scientific progress in NLP centered around developing scalable, data-driven theories of linguistic structure. The idea is to collect data in tightly scoped, carefully defined ways which allow for exhaustive annotation of behavioral phenomena of interest, and then use machine learning to construct explanatory theories of these phenomena which can form building blocks for intelligible AI systems. After laying some conceptual groundwork, I describe several investigations into data-driven theories of shallow semantic structure using Question-Answer driven Semantic Role Labeling (QA-SRL), a schema for annotating verbal predicate-argument relations using highly constrained question-answer pairs. While this only scratches the surface of the complex language behaviors of interest in AI, I outline principles for data collection and theoretical modeling which can inform future scientific progress. This note summarizes and draws heavily on my PhD thesis.


Hey C-Suite: AI Won't Save You!

#artificialintelligence

This article is a collaboration with David Gossett, Principal with Infornautics, who builds first mover technologies that have no instruction set and need to be invented from scratch. He believes data has a story to tell if we apply the right machine models. His specialty is unstructured data. This article is intended to be provocative, to summon curiosity into the issues that plague us today when it comes to machine learning. Three years ago, I wrote this article, Artificial Intelligence Needs to Reset. The AI Hype that was supposed to transpire into all-things automated is still far off. Since that time, we've experienced speed bumps that have pointed to issues including lack of model accountability (black boxes), bias, lack of data representation in the training set etc. An AI Ethics movement emerged to demand more responsible tech, increased model transparency and verifiable models that do what they're supposed to do without impairment or harm to individuals or groups, in the process. Our future is Artificial Intelligence. It's been conjectured that this wonderful AI will be our savior.


The Self-Driving Car: Crossroads at the Bleeding Edge of Artificial Intelligence and Law

McLachlan, Scott, Kyrimi, Evangelia, Dube, Kudakwashe, Fenton, Norman, Schafer, Burkhard

arXiv.org Artificial Intelligence

Artificial intelligence (AI) features are increasingly being embedded in cars and are central to the operation of self-driving cars (SDC). There is little or no effort expended towards understanding and assessing the broad legal and regulatory impact of the decisions made by AI in cars. A comprehensive literature review was conducted to determine the perceived barriers, benefits and facilitating factors of SDC in order to help us understand the suitability and limitations of existing and proposed law and regulation. (1) existing and proposed laws are largely based on claimed benefits of SDV that are still mostly speculative and untested; (2) while publicly presented as issues of assigning blame and identifying who pays where the SDC is involved in an accident, the barriers broadly intersect with almost every area of society, laws and regulations; and (3) new law and regulation are most frequently identified as the primary factor for enabling SDC. Research on assessing the impact of AI in SDC needs to be broadened beyond negligence and liability to encompass barriers, benefits and facilitating factors identified in this paper. Results of this paper are significant in that they point to the need for deeper comprehension of the broad impact of all existing law and regulations on the introduction of SDC technology, with a focus on identifying only those areas truly requiring ongoing legislative attention.


Significant Wave Height Prediction based on Wavelet Graph Neural Network

Chen, Delong, Liu, Fan, Zhang, Zheqi, Lu, Xiaomin, Li, Zewen

arXiv.org Artificial Intelligence

Computational intelligence-based ocean characteristics forecasting applications, such as Significant Wave Height (SWH) prediction, are crucial for avoiding social and economic loss in coastal cities. Compared to the traditional empirical-based or numerical-based forecasting models, "soft computing" approaches, including machine learning and deep learning models, have shown numerous success in recent years. In this paper, we focus on enabling the deep learning model to learn both short-term and long-term spatial-temporal dependencies for SWH prediction. A Wavelet Graph Neural Network (WGNN) approach is proposed to integrate the advantages of wavelet transform and graph neural network. Several parallel graph neural networks are separately trained on wavelet decomposed data, and the reconstruction of each model's prediction forms the final SWH prediction. Experimental results show that the proposed WGNN approach outperforms other models, including the numerical models, the machine learning models, and several deep learning models.


Avian Influenza (H5N1) Warning System using Dempster-Shafer Theory and Web Mapping

Maseleno, Andino, Hasan, Md. Mahmud

arXiv.org Artificial Intelligence

Based on Cumulative Number of Confirmed Human Cases of Avian Influenza (H5N1) Reported to World Health Organization (WHO) in the 2011 from 15 countries, Indonesia has the largest number death because Avian Influenza which 146 deaths. In this research, the researcher built a Web Mapping and Dempster-Shafer theory as early warning system of avian influenza. Early warning is the provision of timely and effective information, through identified institutions, that allows individuals exposed to a hazard to take action to avoid or reduce their risk and prepare for effective response. In this paper as example we use five symptoms as major symptoms which include depression, combs, wattle, bluish face region, swollen face region, narrowness of eyes, and balance disorders. Research location is in the Lampung Province, South Sumatera. The researcher reason to choose Lampung Province in South Sumatera on the basis that has a high poultry population. Geographically, Lampung province is located at 103040' to 105050' East Longitude and 6045' - 3045' South latitude, confined with: South Sumatera and Bengkulu on North Side, Sunda Strait on the Side, Java Sea on the East Side, Indonesia Ocean on the West Side. Our approach uses Dempster Shafer theory to combine beliefs in certain hypotheses under conditions of uncertainty and ignorance, and allows quantitative measurement of the belief and plausibility in our identification result. Web Mapping is also used for displaying maps on a screen to visualize the result of the identification process. The result reveal that avian influenza warning system has successfully identified the existence of avian influenza and the maps can be displayed as the visualization.